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Abstract 

Building  a  robust  and  effective  human-agent  team  is  diffi¬ 
cult,  as  teams  can  fail  for  many  reasons.  In  this  study,  we 
investigated  the  performance  of  human-agent  teams  with 
divisional  and  functional  team  structure  under  different  lev¬ 
el  of  task  load  uncertainty.  Divisional  teams  show  robust¬ 
ness  against  uncertainly  but  can  lead  to  poorer  performance. 
Three  reasons  related  to  team  coordination  were  identified 
for  the  poor  performance,  namely  duplicate  work,  underuti¬ 
lization  of  resources  and  infrequent  communication.  We 
proposed  a  conceptual  model  to  explain  the  mechanism  of 
team  coordination.  In  future  studies,  we  hope  to  quantify 
this  team  coordination  model,  connect  it  with  team  perfor¬ 
mance,  and  test  the  effect  of  different  solutions  using  the 
model. 

Introduction 

By  relieving  the  operator  from  manual  control  tasks,  en¬ 
hanced  autonomy  enables  operators  to  potentially  work 
with  multiple  agents  and  do  higher-level  cognitive  tasks 
requiring  monitoring,  coordination,  and  decision-making. 
However,  the  required  cognitive  load  for  working  with 
multiple  agents  could  easily  exceed  the  capacity  of  a  single 
operator,  even  with  high  levels  of  automation.  Teams  of 
humans  are  increasingly  called  upon  to  perform  complex 
cognitive  tasks  that  are  less  efficiently  done  or  impossible 
to  be  done  by  an  individual.  Although  teamwork  may  im¬ 
pose  extra  workload  related  to  coordination  and  communi¬ 
cation,  teams  have  the  potential  of  offering  greater  adapta¬ 
bility,  productivity,  and  creativity  than  any  one  individual 
can  offer  (Gladstein,  1984). 

However,  the  benefits  of  teamwork  do  not  always  occur 
naturally,  and  teams  can  fail  for  many  reasons  (Salas  & 
Fiore,  2004).  Factors  such  as  poor  combination  of  individ¬ 
ual  efforts,  a  breakdown  in  internal  team  processes  (e.g., 
communication),  and  improper  use  of  available  infor¬ 
mation  have  been  identified  as  potential  sources  of  team 
failure  (Salas,  Sims,  &  Burke,  2005).  To  achieve  robust 
and  effective  teamwork,  we  must  understand  the  nature  of 
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such  teamwork,  including  structure,  outcomes,  processes 
and  dynamics. 

Team  structure  determines  how  individual  efforts  are 
combined  in  the  team,  which  is  influenced  by  task  assign¬ 
ment  and  coordination  processes.  For  a  team  of  operators 
working  together  to  supervise  a  complex  system  such  as 
multiple  heterogeneous  unmanned  vehicles,  there  are  sev¬ 
eral  ways  to  organize  the  vehicles.  One  common  method  is 
functional  organization,  in  which  individuals  specialize 
and  perform  certain  roles.  For  example,  a  manufacturing 
plant  can  be  organized  by  separating  engineering,  account, 
manufacturing,  personnel  and  purchasing  specialists  into 
different  departments  based  on  their  functions.  Through 
specialization  of  each  member,  groups  are  able  to  tackle 
problems  more  efficiently. 

Another  way  to  organize  the  team  is  divisional  organiza¬ 
tion,  in  which  each  working  unit  can  be  responsible  for  all 
type  of  tasks.  In  divisional  organization,  each  member  is 
allocated  with  some  resources  of  each  type.  By  creating 
self-contained  tasks,  it  reduces  the  amount  of  information 
processed  within  an  organization  when  the  level  of  uncer¬ 
tainty  is  high  (Galbraith,  1974).  For  example,  a  company 
can  have  several  divisions  each  responsible  for  one  prod¬ 
uct.  Each  division  has  its  own  set  of  functional  units  like 
research,  design,  marketing  etc.  Divisional  structure  was 
designed  in  order  to  have  a  fast  response  to  the  market 
(Macmillan,  Entin,  &  Serfaty,  2004).  In  one  command  and 
control  scenario,  it  was  found  that  the  effectiveness  of 
teams  using  the  divisional  and  functional  structures  de¬ 
pends  on  the  nature  of  the  tasks  to  be  accomplished  and  the 
uncertainty  in  the  situation.  Specifically,  functional  teams 
perform  better  when  the  environment  and  tasks  are  predict¬ 
able.  Divisional  teams  have  a  higher  level  of  robustness 
and  perform  better  when  the  environment  and  tasks  have 
more  uncertainty  (Macmillan,  Entin,  &  Serfaty,  2004). 

Communication,  an  important  coordination  mechanism, 
influences  the  share  of  information  among  team  members. 
Over  the  course  of  a  conversation,  units  of  shared  infor¬ 
mation  are  transformed  into  mutual  team  knowledge, 
which  is  the  foundation  of  successful  team  coordination. 
Team  knowledge  consists  of  background  knowledge  that  is 
long-lived  in  nature,  as  well  as  more  dynamic  and  fleeting 
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understanding  that  an  operator  has  of  a  situation  at  any  one 
point  in  time  (Cooke,  Kiekel,  &  Helm,  2001).  It  is  influ¬ 
enced  by  team  process  behaviors,  such  as  communication, 
coordination  and  situation  awareness  (Cooke,  et  ah,  2001). 
If  the  team  members  don’t  have  a  clear  understanding  of 
the  situation,  it  may  result  in  delayed  actions,  errors,  and  a 
suboptimal  distribution  of  team  resources. 

Communication  also  relates  to  team  trust,  which  indi¬ 
rectly  influences  team  performance.  Previous  research 
found  that  teams  with  different  levels  of  trust  also  have  dif¬ 
ferent  communication  styles.  Team  members  with  the 
highest  levels  of  trust  were  sociable,  exchanged  frequent 
messages,  showed  interest  in  other  members’  responses, 
showed  initiative,  provided  substantive  feedback  to  one 
another,  and  notified  others  of  their  expected  participation 
periods  or  absences.  Those  with  the  lowest  levels  of  trust 
exhibited  little  initiative  and  had  little  social  content  in 
their  messages.  Groups  with  moderate  trust  levels  had  pre¬ 
dictable  but  infrequent  communication,  focusing  their  mes¬ 
sages  on  tasks  only  and  devoting  a  disproportionate  level 
of  messages  to  establishing  rules  and  procedures 
(Jarvenpaa  &  Leidner,  1998).  Similarly,  Dirks  (1999) 
found  low  trust  teams  worked  more  as  individuals  whereas 
high  trust  teams  exhibited  behaviors  more  consistent  with 
joint  effort.  Thus,  trust  indirectly  influences  group  perfor¬ 
mance  by  moderating  the  relationship  between  motivation 
and  group  process  and  performance. 

Experimental  Data  Analysis 

Human-agent  teams  often  work  under  uncertainty.  One 
major  source  of  uncertainty  is  task  load.  The  arrival  time 
and  types  of  tasks  are  often  unpredictable.  To  achieve 
adaptability  and  robustness,  teams  should  be  able  to 
properly  assign  the  tasks  and  balance  the  workload  among 
team  members.  In  this  study,  we  investigated  the  commu¬ 
nication  and  coordination  process  and  performance  of  hu¬ 
man-agent  teams  with  different  team  structures  and  under 
different  levels  of  task  load  uncertainly. 

An  experimental  test-bed  where  teams  of  operators  con¬ 
trolled  multiple  unmanned  vehicles  (UVs)  was  developed 
and  experimental  trials  were  conducted  to  gather  the  data 
(Figure  1).  The  main  part  of  the  interface  is  the  map  with 
the  locations  of  contacts  and  vehicles.  The  operators  were 
able  to  communicate  with  each  other  via  instant  messaging 
in  a  chat  interface  window.  The  System  Panel  presented 
system  messages  on  vehicle  status,  conflicts  of  task  as¬ 
signments,  etc.  Operators  could  observe  the  positions  of 
vehicles  controlled  by  their  teammates  by  clicking  and 
holding  the  “Monitor  Other  Vehicles”  button,  as  seen  in 
Figure  1. 


The  simulation  included  three  ground  control  stations, 
with  one  subject  assigned  to  each  station.  The  three  opera¬ 
tors  were  referred  as  Alpha  (A),  Bravo  (B),  and  Charlie 
(C).  Each  mission  scenario  required  a  team  of  operators  to 
“handle”  contacts  that  appeared  intermittently  over  the 
map,  requiring  operators  to  perform  both  assignment  and 
payload  tasks.  Assignment  tasks  required  the  operators  to 
send  their  vehicles  to  the  contacts  on  the  map  as  they 
emerged.  Once  a  vehicle  reached  a  contact,  the  operator 
identified  the  contact,  and  then  destroyed  the  contact  or 
dropped  aid  packages,  depending  on  its  hostile  or  friendly 
status. 


Figure  1:  Team-RESCHU  Interface 


The  tasks  to  identify,  destroy  or  drop  aid  packages  were 
completed  using  three  different  types  of  vehicles.  In  other 
words,  two  vehicles  were  required  to  complete  one  task. 
The  second  vehicle  could  be  assigned  before  or  after  the 
identification.  The  timeline  for  processing  a  task  is  shown 
in  Figure  2.  The  time  between  the  appearance  of  an  un¬ 
known  contact  and  the  time  it  was  destroyed  or  aided  was 
called  objective  completion  time.  Team  performance  was 
measured  by  averaging  the  objective  completion  time  of 
each  contact  during  the  mission. 

A  2x2  repeated  measures  experiment  was  conducted 
where  the  independent  variables  were  team  structure  (divi¬ 
sional,  functional)  and  the  inter-arrival  time  of  unidentified 
contacts  (constant,  erratic).  Ten  teams  of  three  participants 
each  completed  all  four  treatments.  The  experimental  trials 
had  a  total  of  16  exogenous  events  (unidentified  contacts 
emerging).  The  time  between  successive  exogenous  events 
(the  inter- arrival  time)  was  30  seconds  for  the  constant 
treatment.  For  the  erratic  factor  level,  the  inter-arrival 
times  were  generated  from  a  bimodal  distribution  where 
the  means  of  the  modes  were  set  at  75  seconds  and  225  se¬ 
conds  from  the  start  of  the  trial,  with  a  standard  deviation 
of  15  seconds. 
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Figure  2:  Timeline  for  Processing  a  Task 


The  second  independent  variable  was  team  structure.  A 
functional  team  is  one  where  the  operators  have  rigidly  de¬ 
fined  roles  and  responsibilities.  For  instance,  when  all  of 
the  vehicles  of  one  type  are  assigned  to  one  and  only  one 
operator,  then  that  operator  is  given  the  full  responsibility 
for  performing  the  tasks  that  only  that  vehicle  can  do.  If 
one  of  each  vehicle  type  is  allocated  to  a  single  operator 
instead,  then  that  team  structure  would  be  considered  divi¬ 
sional  since  any  operator  can  perform  any  task  that  arises, 
provided  that  he  has  an  appropriate  vehicle  available. 

The  initial  experiment  results  showed  that  functional 
teams  performed  significantly  better  than  divisional  teams 
in  general.  Functional  teams  performed  better  with  con¬ 
stant  arrival,  while  divisional  teams  perform  better  with  er¬ 
ratic  arrival  (Mekdeci  &  Cummings,  2009).  Although  divi¬ 
sional  teams  showed  their  robustness  against  the  uncertain¬ 
ty  of  task  arrival,  their  performance  was  not  as  good  as  de¬ 
sired.  In  this  effort,  we  further  investigated  the  teamwork 
process  to  identify  several  reasons  for  the  poor  perfor¬ 
mance:  duplicate  work,  underutilization  of  vehicles,  and 
infrequent  communication. 

Duplicate  Work 

Further  analysis  on  teamwork  process  shows  that  teams 
that  have  worse  performance  also  tend  to  have  bad  team 
coordination.  One  example  of  such  poor  coordination  oc¬ 
curred  in  divisional  teams,  where  we  observed  duplicate 
vehicle  assignment.  As  shown  in  Figure  3,  vehicles  of  the 
same  type  controlled  by  different  operators  were  assigned 
to  the  same  task,  which  resulted  in  a  waste  of  resources. 


Task  1-16 

Figure  3:  Vehicles  Assigned  to  Each  Task  by  Operators 


This  requires  conflict  resolution  within  the  team  by  ex¬ 
plicit  communication,  which  cost  time.  In  the  communica¬ 
tion  transcript,  we  observed  messages  such  as  these: 

Bravo:  B  1 

Alpha:  A  1 

Charlie:  Let  B  take  it  -  he’s  closer,  A  you  take  2 

Alpha:  Redirecting  to  2 

We  can  see  that  both  Bravo  and  Alpha  wanted  to  work 
on  contact  1.  This  conflict  was  resolved  via  communica¬ 
tion  as  Charlie  asked  Bravo  to  work  on  contact  1  and  Al¬ 
pha  on  contact  2.  Communication  was  necessary  in  this 
case  but  cost  extra  time.  If  the  conflict  is  not  resolved,  it 
may  happen  that  some  tasks  have  multiple  operators  work¬ 
ing  on  them  while  others  are  ignored. 

Under  Utilization  of  Vehicles 

A  second  reason  for  the  poor  performance  of  divisional 
teams  is  the  under  utilization  of  the  vehicles.  Figure  4 
shows  the  working  process  of  a  divisional  team  with  con¬ 
stant  task  arrival.  Each  column  is  the  timeline  of  one  emer¬ 
gent  contact  from  its  appearance  until  it  is  destroyed  or 


provided  aid  packets.  Green  is  for  vehicle  travel  time,  yel¬ 
low  is  for  identification  time,  red  is  for  destroy  time,  and 
blue  is  for  rescue  time.  Dark  grey  is  for  idle  time,  during 
which  the  target  is  not  assigned  a  vehicle. 

The  dark  grey  period  is  nonproductive,  which  happens  a 
lot  for  this  team.  Contacts  are  numbered,  with  a  letter  T 
added  after  the  identification  for  threats  or  F  for  friendly. 
For  example,  1  is  an  unidentified  contact.  It  is  updated  as 
IT  if  identified  as  hostile  or  IF  if  friendly.  The  longest  idle 
time  was  highlighted  by  the  black  box  in  Figure  4.  We 
looked  at  the  log  of  communication  and  the  actions  of  op¬ 
erators  during  this  time,  which  are  summarized  in  Table  1. 

Operator  Alpha  assigned  a  vehicle  to  an  unknown  con¬ 
tact  4,  and  later  to  a  threat  contact  OT  and  a  friendly  con¬ 
tact  3F.  All  three  vehicles  operated  by  Alpha  were  busy. 
He  also  reported  his  action  to  his  teammates  via  chat  mes¬ 
sages.  Operator  Bravo  was  working  on  threat  contact  2T. 
After  that,  he  worked  on  another  threat  contact  4T,  and  lat¬ 
er  assigned  a  vehicle  to  an  unknown  contact  7.  Only  one 
vehicle  controlled  by  Bravo  was  busy  at  one  time.  Operator 
Charlie  assigned  vehicles  to  several  contacts  (4,  3F,  4T, 
and  OT)  after  he  finished  identifying  unknown  contact  3. 
However,  all  of  these  contacts  had  already  been  claimed  by 
the  other  two  team  members.  He  then  asked  about  task  al¬ 
location  and  assigned  a  vehicle  to  contact  IF.  During  this 
time,  none  of  the  vehicles  operated  by  Charlie  were  busy 
and  none  of  them  were  assigned  to  contact  5. 
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Figure  4:  Timeline  of  Task  Completion  in  a  Divisional  Team 
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Table  1:  Coordination  and  Vehicle  Utilization 


From  these  we  can  see  the  vehicles  were  not  used  at  their 
full  capacity.  While  there  were  enough  idle  vehicles,  some 
tasks  had  no  vehicle  assigned  to  them. 

Infrequent  communication 

We  found  that  chat  density  had  an  influence  on  the  task  as¬ 
signment  waiting  time,  which  is  the  nonproductive  time 
between  the  appearance  of  a  contact  and  the  time  it  was  as¬ 
signed  a  vehicle.  We  conducted  a  partial  correlation  analy¬ 
sis  for  average  objective  completion  time,  average  assign¬ 
ment  waiting  time,  and  the  number  of  chat  messages.  Team 
structure,  arrival  process,  and  trial  sequence  were  con¬ 
trolled  in  order  to  separate  the  influence  of  communication. 
Although  chat  density  did  not  have  a  significant  correlation 
with  the  overall  objective  completion  time,  it  negatively 
correlated  with  average  assignment  waiting  time  with  r  =  - 
0.427,  p  =  0.009.  Average  assignment  waiting  time  corre¬ 
lated  with  average  objective  completion  time  with  r  = 
0.3923,  p  =  0.016.  In  other  words,  communication  indi¬ 
rectly  influenced  team  performance  by  reducing  the  non¬ 
productive  time.  Thus  teams  that  communicated  infre¬ 
quently  likely  led  to  poorer  performance. 

A  Conceptual  Model  of  Team  Coordination 

In  order  to  explain  these  barriers  systematically,  we  pro¬ 
pose  a  conceptual  model  for  team  coordination  as  shown  in 
Figure  5.  Team  Knowledge  is  represented  as  a  matrix  K 
where  Ktj-  stands  for  Operator  z’s  understanding  of  Opera¬ 
tor  y’s  situation,  such  as  the  availability  of  vehicles  vtj  and 
the  distance  to  contacts  dtj.  Operator  i  makes  a  decision  on 
whether  to  work  on  a  task  based  on  his/her  understanding 
of  the  situation,  which  is  represented  by  the  matrix  K.  Spe¬ 
cifically,  x>ij  takes  a  value  of  1  if  the  vehicle  is  idle  or  oo  if 
it  is  working.  Let  Ktj  =  min{  v^d^}  for  the  multiple  vehi¬ 
cles  with  the  matched  capability  for  a  task  controlled  by 
Operator  y,  Operator  i  will  decide  to  work  on  the  task  if 
Ku  <  min for  all  j  =£  i.  In  other  words,  Operator  i 
would  decide  to  work  on  a  task  if  he/she  thought  his/her 
vehicle  was  the  closest  available  vehicle. 

After  a  decision  is  made  or  an  action  is  taken,  the  opera¬ 
tor  may  communicate  with  the  teammates  by  reporting 
his/her  intention,  which  updates  column  i  of  the  matrix. 
Conflict  resolution  may  be  necessary  if  another  operator 
has  already  taken  the  task,  which  leads  to  a  change  of  task 
assignment  action.  If  Operator  i  requests  for  information 
from  his/her  teammates  or  gets  information  implicitly,  row 
i  of  the  matrix  is  updated.  An  example  of  implicit  infor¬ 
mation  sharing  is  that  Operator  i  presses  the  ‘Monitor  other 
Vehicles’  button  (Figure  1)  to  show  the  positions  of  his/her 
teammates’  vehicles  on  the  map  in  Team-RESCHU. 


Team  Knowledge 


Figure  5:  A  Conceptual  Model  of  Team  Coordination 


However,  this  coordination  process  may  not  be  as 
straightforward  as  depicted.  Problems  may  happen  at  each 
step  of  the  process.  Team  members  may  not  communicate 
enough  about  their  actions  and  intensions.  Even  if  they  do, 
communication  messages  may  not  be  noticed  instantly  or 
even  be  ignored,  which  leads  to  biased  and  outdated  team 
knowledge.  Duplication  of  task  assignment  can  happen  if 
the  task  assignment  actions  are  decided  based  on  such 
flawed  information,  but  conflict  resolution  may  not  always 
be  triggered.  The  update  of  team  knowledge  via  infor¬ 
mation  requests  or  implicit  information  sharing  often  re¬ 
quires  extra  time  and  efforts,  which  may  extend  the  task 
time  and  increase  the  workload. 

The  purpose  of  the  model  is  to  both  explain  why  team 
coordination  fails,  but  also  to  develop  solutions  to  prob¬ 
lems  at  each  step.  For  example,  improving  information 
display  on  the  interface  can  enhance  implicit  information 
sharing.  For  communication,  building  successful  team  co¬ 
ordination  may  aid  in  building  team  trust.  Behaviors  such 
as  communicating  frequently,  acknowledging  others,  and 
being  explicit  was  found  to  significantly  correlate  with 
team  trust  (Walther,  Bunz,  &  Bazarova,  2005).  These  be¬ 
haviors  also  improve  team  coordination,  and  may  influence 
team  performance  indirectly. 

Conclusion 

Effective  teamwork  in  highly  dynamic  environments  re¬ 
quires  a  delicate  balance  between  giving  agents  the  auton¬ 
omy  to  act  and  react  on  their  own  and  restricting  that  au¬ 
tonomy  so  that  the  agents  do  not  work  at  cross  purposes 
(Work,  Chown,  Hermans,  &  Butterfield,  2008).  In  this 
study,  divisional  teams  were  designed  to  create  working 
units  that  have  a  higher  level  of  autonomy.  They  show  ro¬ 
bustness  against  uncertainly  but  poorer  performance.  Three 
reasons  related  to  team  coordination  were  identified  for  the 
poor  performance,  namely  duplicate  work,  underutilization 
of  resources  and  infrequent  communication.  We  proposed 
a  conceptual  model  to  explain  the  mechanism  of  team  co¬ 
ordination.  In  future  studies,  we  hope  to  quantify  this  team 
coordination  model,  connect  it  with  team  performance,  and 
test  the  effect  of  different  solutions  using  the  model. 
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